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Stage  Setting 


The  PM  needs  visibility  across  multiple  projects  within 
his  family-of-systems 

Data  is  managed  in  various  systems,  files  and  databases 

Available  data  was  in  various  formats 

Base  measures  were  available  for  some  needs,  but 
calculations  were  needed  to  obtain  some  derived 
measures 

Not  all  needed  base  measures  were  available 


C  Cask 
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'he  Detour 


Organizational 

Detour: 

Mountains  of  data 
that  fail  to  provide 
actionable 
information 


Cultural  Detour: 
Actionable  information 
is  available  but  it’s  not 
used 


Technical  Detour: 
Tool  implementation 
that  doesn't  meet 
organizational 
needs 
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Competitive  Advantage 


Measurement  Continuum 


k 

Optimization 

What's  the  best  that  can  happen? 

Predicti  Modeling 

What  will  happen  next? 

Forecasting/Extrapolation 

What  if  these  trends  continue? 

Statistical  Analysis 

Why  is  this  happening? 

Alerts 

What  actions  are  needed? 

Query/Drill  Down 

Where  exactly  is  the  problem? 

Ad  Hoc  Reports 

How  many,  how  often,  where? 

Standard  Reports 

- ^ 

What  happened?  _ , 

Analytics 


Reporting 


Degree  of  Intelligence 


C  Cask 


Source:  Competing  on  Analytics,  Davenport  and  Harris,  2007 
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Measurement  Capability  Roadmap 


Assess  Existing  Data  and  Analytical  Capabilities 

•  Assess  analytical  capabilities 

•  Assess  existing  data  and  alignment 


Implement  Small  Project  and  Document  Benefits 

•  Find  a  sponsor  and  business  problem 

•  Implement  a  small,  localized  project 

•  Document  the  costs  and  benefits 

Develop  Enterprise  Metrics  and  Architect  Solutions 

•  Define  and  manage  a  set  of  achievable  performance  metrics 

•  Align  analytical  resources  and  establish  timetable 

•  Architect  technology  solutions 

Implement  Enterprise  Solution  and  Organization  Changes 

•  Execute  measurement  projects 

•  Align  measurement  resources  across  the  organization 

•  Manage  organizational  changes 

Focus  on  Continuous  Improvement 

•  Continuous  benefits 

•  Focus  on  continual  improvement 


Cask, 
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Implementation  Challenges 


Organization 

•  The  right  information  to  the  right  people  at  the  right  time 

•  Performance  management  and  strategy  execution 

•  Process  redesign  and  integration 

Culture 

•  Leadership  and  senior  executive  commitment 

•  Establishing  a  fact-based  culture 

•  Securing  and  building  skills 

•  Managing  analytical  people 

Technology 

•  Quality  data 

•  Analytic  technologies 


C  Cask 
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Increased  Capability 


Capability  Assessment 


Small-scale 
Implementation 

Enterprise  Solutions 

Enterprise 
Implementation 

Continuous 
V  Improvement 


Alignment 


Roadmap  Stage 

Organizational 

Cultural 

Technical 

Assessment 

Autonomous  Efforts 

Coordinated  Planning 
Coordination  Execution 


C  Cask 
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Organizational  Challenges 


Information  Need 


•The  right  information  to  the  right 
people  at  the  right  time 
•Performance  management  and 
strategy  execution 
•Process  redesign  and  integration 


Information  Product 


■4- 


^^^nterpretatiorT^^ 


ISO/IEC  15939 


C  Cask 


Measurement 

Capability 


Information  Need  & 
Measurement  ^ 
Objectives  Alignment 


Collection  & 
Results 
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What  Does  the  PM  Need  to  Know? 


Goal  1:  Meet  the  needs  of  the  end  user  and  the  stakeholder  community 
Goal  2:  Enhance  financial  management  and  execution  oversight 
Goal  3:  Improve  acquisition,  supportability  and  engineering  processes 
Goal  4:  Develop  core  acquisition  functions  among  the  workforce 

Questions: 

•  Are  the  projects  on  track? 

•  Are  the  projects  on  schedule? 

•  Do  projects  have  approved  requirements?  What  is  the  status  of  a  project's  requirements? 

•  What  is  the  degree  of  risk  associated  with  each  project?  Which  projects  are  most  at  risk? 

•  Who  is  supporting  a  project?  Is  there  adequate  staff?  Is  the  staff  adequately  skilled? 

•  How  many  of  each  type  of  project  is  in  the  portfolio?  How  many  projects  are  in  each  major  phase? 

•  Does  the  project  have  sufficient  money  to  conduct  acquisition  activities  on  this  project? 

•  What  is  the  current  funding  status?  How  do  we  compare  against  OSD  and  FMB  Benchmarks? 

•  What  are  the  current  year  funding  deficiencies?  By  cause?  By  project?  By  impact? 

•  What  is  the  value  of  contracts  that  are  ending  in  the  next  quarter,  half  year? 

•  What  is  the  value  of  current  contracts  for  each  team? 

•  What  are  the  values  of  the  contracts  each  project  officer  is  managing? 

©  201 1  -  Cask,  LLC  |  Strategy.  Solutions.  Success. 
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Example  Goal-Question-Indicators 


Questions 

Goal 

Indicator 

Measures 

Are  the  projects  on  track? 

2 

Milestone  Completion 

•  Milestone  Progress 

•  Interim  Progress 

•  Trend 

What  is  the  degree  of  risk  associated  with 
each  project?  Which  projects  are  most  at 

risk? 

1 

Risk  Status 

•  Risk  Likelihood 

•  Risk  Impact 

Are  the  projects  on  schedule? 

1 

Milestone  Completion 
Work  Unit  Progress 

•  Milestone  Dates 

•  Test  Cases  Passed 

•  Requirements  Tested 

•  Reviews  Completed 

What  is  the  current  funding  status?  How  do 
we  compare  against  OSD  and  FMB 
Benchmarks? 

2 

Financial  Adequacy 

•  Obligation  Rates 

•  Disbursement  Rates 

•  Funding  Availability 

Has  the  program  office  established  realistic 

cost  and  schedule  estimates  for  the 
projects? 

1 

Schedule  Feasibility 

Cost  Feasibility 

•  Schedule  Probability 

•  Cost  Probability 

Do  the  projects  have  sufficient  money  to 
conduct  acquisition  activities? 

2 

Financial  Performance 

•  Cost 

•  BCWS,  BCWP,  ACWP 

®  Cask. 
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•Senior  executive  commitment 


Human  Challenges 


•Establishing  a  fact-based  culture 
•Securing  and  building  skills 


•Managing  analytical  people 
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Solution:  Execute  Organizational  Change  Management  Activities  in  Alignment 

with  Measurement  Roadmap 


Assessment 

Small  Scale  Implementation 

Enterprise  Planning 

Enterprise  Implementation 

Cl 

Enabling 

Activities 


Engaging 

Activities 


Make  the  Case 


Build  the  Team 


Define  the  End  State 


Communicate  for  Buy-In 


Get  Action 


Generate  Momentum 


Sustaining 

Activities 


Maintain  Momentum 


Solidify  Gains 
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SCask. 


Key  Items  to  Consider 

•  Gaining  alignment  with  executive 
sponsorship 

•  Cascading  alignment  through  the 
organizational  structure 

•  Manage  concerns  and  capabilities  among 
the  workforce 


JCask^ 
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Key  Role  Map  is...  well  key 


\ 
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Technology  Challenges 


•Quality  data 

•Ability  to  share  information 
•Measurement  and  analytic 
technologies 
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Solution:  Mature  and  Architect  Technology  in  Accordance  with  Needs  Specific 
to  Each  Stage  of  the  Measurement  Roadmap 


•  Assessment:  Understand  performance  measurement  needs, 
determine  information  needs,  align  to  organizational  strategy 

•  Small-scale  Implementation:  Align  information  needs, 
measurement  functions  and  measureable  entities 

•  Enterprise  Solutions:  Standardize  data  and  technology 
governance 

•  Enterprise  Implementation:  Establish  and  manage  technology 
architecture 

•  Continuous  Improvement:  Technology  refresh  and  upgrades 


JCask^ 
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Why  Not  the  Simple  Solution? 


We  know  it's  a 
natural  choice. 


BUT 


•  •  • 


s 


No  learning  curve 


Available  &  easy 
to  use 


r 

Difficult  to  access 
easily 

L  A 


Burdensome  data 
integrations 


r  ^ 

Macros  may  help 

L  A 


Hard  to  support 
multiple  users 
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Problems  Implementing  Measurement 


Establish 
Mgmt  policy 


C  Cask, 


Generate 

scripts 


What  happens  when 
the  policy,  script  or 
graphs  change? 


It  stops  working! 


Produce 

4 

^  4 

charts 

Collect  & 
validate  data 


©2011  - 
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Why  DataDrill? 


18 


DataDrill  EXPRESS  saves  time  and  money, 
and  gets  better  results. 


Microsoft  Excel 


DataDrill  EXPRESS 


C  Cask, 
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DataDrill  Express  Components 


Managers 


Tf  7TT 


A 
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CD 

U 
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Administrators 


/  1  1  v 


j 

j 

j 
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j 

j 

< 

i 

< 
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i 
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Executives 

EXPRESS  Web  Client 
license  types 


EXPRESS  Server 


Web 
Services 


Application 

Interfaces 

Microsoft 

HP 

IBM/Rational 

IBM/Telelogic 


EXPRESS  Library 


EXPRESS  Portal 


X 


Measurement 

Repository 

t 

EXPRESS  Collectors 


i  I 


I 


SQL 

Interfaces 

Oracle 
SQL  Server 
ODBC 


General 
Interfaces 

CSV 
Excel 
XML 
Access 


Management 

Activities 


Data 

Collection 
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Sample  Indicator 
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Team  Project  Appropriation 


Funding 


Reference 


Team  A 


Project  A  PMC 


RDTE  C9999A 

OMMC  XAXA 

Project  A  Total 


Year  Authorized 


2009  $5,777,128 

2010  $1,285,000 
2011  $4,515,000 
2011  $1,028,000 
2011  $2,509,394 

$15,114,522 


Project  B  RDTE 
OMMC 

Project  B  Total 

Project  C  PMC 
RDTE 
OMMC 

Project  C  Total 

Project  D  PMC 

Project  D  Total 


C9999F  2010  $1,622,400 

2011  $1,133,000 
XAXA  2011  $1,715,000 

$4,470,400 

999999  2009  $4,723,872 

C9999B  2011  $502,000 

XAXA  2011  $74,803 

$5,300,675 

999999  2010  $1,719,000 

$1,719,000 


Financial  Adequacy 


Committed 

Obligated 

%  CMT 

%  OBL 

UNOBL 

AMT 

%OBL  OSD 

Benchmark 

OSD  OBL 

Benchmark 

Status 

%OBL  FMB 

Benchmark 

FMB  OBL 

Benchmark 

Status 

$5,300,626 

$5,743,566 

91.75% 

99.42% 

$33,562 

91.7% 

7.7% 

92.0% 

7.4% 

$1,024,840 

$1,024,840 

79.75% 

79.75% 

$260,160 

81.7% 

-1.9% 

82.0% 

-2.2% 

$598,500 

$596,545 

13.26% 

13.21%  $3,918,455 

13.3% 

-0.1% 

6.0% 

7.2% 

$144,200 

$144,200 

14.03% 

14.03% 

$883,800 

15.0% 

-1.0% 

31.0% 

-17.0% 

$65,570 

$65,570 

2.61% 

2.61%  $2,443,824 

16.7% 

-14.1% 

21.3% 

-18.7% 

$7,133,736 

$7,574,721 

47.20% 

50.12%  $7,539,801 

$1,424,222 

$1,424,222 

87.78% 

87.78% 

$198,178 

91.7% 

-3.9% 

95.8% 

-8.0% 

$169,950 

$169,950 

15.00% 

15.00% 

$963,050 

15.0% 

0.0% 

31.0% 

-16.0% 

$980,000 

$72,797 

57.14% 

4.24%  $1,642,203 

16.7% 

-12.5% 

21.3% 

-17.1% 

$2,574,172 

$1,666,968 

57.58% 

37.29%  $2,803,432 

$4,332,000 

$4,332,000 

91.70% 

91.70% 

$391,872 

91.7% 

0.0% 

92.0% 

-0.3% 

$75,300 

$74,000 

15.00% 

14.74% 

$428,000 

15.0% 

-0.3% 

31.0% 

-16.3% 

$48,803 

$48,803 

65.24% 

65.24% 

$26,000 

16.7% 

48.5% 

21.3% 

43.9% 

$4,456,103 

$4,454,803 

84.07% 

84.04% 

$845,872 

$1,118,994 

$984,286 

65.10% 

57.26% 

$734,714 

81.7% 

-24.4% 

82.0% 

-24.7%  1 

$1,118,994 

$984,286 

65.10% 

57.26% 

$734,714 
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Sample  Indicator  -  Requirements  Volatility 


Requirements  Volatility 


Vblatlity  Aarm 


sio  4no  5}  to  &«io  7.uo  aio  aria  ton  a  imo  is  to 


Added  Requirements  |  Deleted  Requirements  |  Edited  Requirements  Planned  Requirements  Total  Requirements  — Volatility  Percentage 


1 
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Summary 


22 


•  The  implementation  of  a  measurement  capability 
provides  both  reporting  and  analytical  capabilities 

•  A  well-defined  roadmap  for  implementing  a 
measurement  capability  provides  a  disciplined  approach 

•  This  disciplined  approach  addresses  process, 
organizational  and  technical  challenges 


JCask^ 
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List  of  Acronyms 


•  ISO/IEC  15939  -  International  Organization  for  Standardization/International 
Electrotechnical  Commission:  System  and  Software  Engineering  Measurement  Process 

•  Cl  -  Continuous  Improvement 

•  DIR/S  -  Director/Sponsor 

•  PGM/S  -  Program  Manager/Sponsor 

•  TL/S  -  Team  Lead/Sponsor 

•  PM/T  -  Project  Manager/Target 

•  ODBC  -  Open  Database  Connectivity 

•  XML  -  Extensible  Markup  Language 

•  CSV  -  Comma  Separated  Values 

•  PMC  -  Procurement 

•  RDTE  -  Research,  Development,  Test  and  Evaluation 

•  OMMC  -  Operation  and  Maintenance 

•  %  CMT  -  Percent  Committed 

•  %  OBL  -  Percent  Obligated 

•  UNOBL  AMT  -  Unobligated  Amount 

•  OBL  OSD  Benchmark  -  Office  of  the  Secretary  of  Defense  Obligation  Benchmark 

•  OBL  FMB  Benchmark  -  Financial  Management  Branch  Obligation  Benchmark 
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